Imaging apparatus, image processing apparatus, and image processing method for generating random special effects

ABSTRACT

An imaging apparatus includes an imaging unit, a random seed generating unit, a pseudo-random number generating unit, and a special image processing unit. The imaging unit photographs a subject to obtain image data. The random seed generating unit generates a random seed to decide a pseudo-random number sequence. The pseudo-random number generating unit generates a pseudo-random number in accordance with the generated random seed. The special image processing unit performs special image processing to apply a special effect to the image data based on the generated pseudo-random number. The random seed is decided during photography.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2013-094412, filed Apr. 26,2013, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus and an imageprocessing apparatus capable of special image processing, and an imageprocessing method therefor.

2. Description of the Related Art

There have been developed a large number of imaging apparatuses havingthe function of subjecting image data obtained by photography to specialimage processing. A large number of techniques associated with thespecial image processing have also been suggested. For example, Jpn.Pat. Appln. KOKAI Publication No. 2010-62836 has suggested a method ofgenerating a high-contrast image which has a granular feeling (noisefeeling) as in a film image. The method according to Jpn. Pat. Appln.KOKAI Publication No. 2010-62836 enables the photography of still imagesand moving images having a rough and strong impression. Moreover, Jpn.Pat. Appln. KOKAI Publication No. 2010-74244 has suggested a method ofgenerating an image which is darkened in its periphery. The methodaccording to Jpn. Pat. Appln. KOKAI Publication No. 2010-74244 enablesthe photography of still images and moving images that seem to have beenphotographed with a toy camera.

BRIEF SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided animaging apparatus comprising: an imaging unit which photographs asubject to obtain image data; a random seed generating unit whichgenerates a random seed to decide a pseudo-random number sequence; apseudo-random number generating unit which generates a pseudo-randomnumber in accordance with the generated random seed; and a special imageprocessing unit which performs special image processing to apply aspecial effect to the image data based on the generated pseudo-randomnumber, wherein the random seed is decided during photography.

According to a second aspect of the invention, there is provided animage processing apparatus comprising: a random seed generating unitwhich generates a random seed to decide a pseudo-random number sequence;a pseudo-random number generating unit which generates a pseudo-randomnumber in accordance with the generated random seed; and a special imageprocessing unit which performs special image processing to apply aspecial effect to image data based on the generated pseudo-randomnumber, wherein the random seed is decided during the acquisition of theimage data.

According to a third aspect of the invention, there is provided an imageprocessing method comprising: generating a random seed by use ofinformation obtained during an acquisition of image data in response toan instruction to perform special image processing for the image data;generating a pseudo-random number in accordance with the generatedrandom seed; and performing special image processing to apply a specialeffect to the image data on based on the generated pseudo-random number.

According to a fourth aspect of the invention, there is provided anon-transitory recording medium on which an image processing programcausing a computer to execute: generating a random seed by use ofinformation obtained during an acquisition of image data in response toan instruction to perform special image processing for the image data;generating a pseudo-random number in accordance with the generatedrandom seed; and performing special image processing to apply a specialeffect to the image data on based on the generated pseudo-random number.

Advantages of the invention will be set forth in the description whichfollows, and in part will be obvious from the description, or may belearned by practice of the invention. The advantages of the inventionmay be realized and obtained by means of the instrumentalities andcombinations particularly pointed out hereinafter.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the general description given above and the detaileddescription of the embodiments given below, serve to explain theprinciples of the invention.

FIG. 1 is a block diagram showing the configuration of a digital cameraas an example of an imaging apparatus comprising an image processingapparatus according to one embodiment of the present invention;

FIGS. 2A and 2B are flowcharts showing the main operation of the digitalcamera according to one embodiment of the present invention;

FIG. 3 is a flowchart showing effect setting processing;

FIG. 4 is a flowchart showing image processing;

FIG. 5 is a flowchart showing basic image processing;

FIG. 6 is a flowchart showing special image processing;

FIG. 7 is a diagram showing scratch image data;

FIG. 8 is a diagram showing noise image data;

FIG. 9 is a diagram showing dust image data;

FIG. 10 is a flowchart showing processing to apply a film noise effect;

FIG. 11 is a graph showing an example of the relation of a thresholdused in a determination in step S602 with the number of frames;

FIG. 12A is a diagram showing an overview of the update of a cutoutposition of the scratch image data;

FIG. 12B is a diagram showing an overview of a fine correction of thecutout position of the scratch image data;

FIG. 12C is a diagram showing an overview of the correction of thecutout position of the scratch image data after the cutout position hasreached an upper end;

FIG. 13 is a graph showing an example of the relation between the numberof frames and the threshold for making a determination in step S606;

FIG. 14 is a diagram showing an overview of the update of a cutoutposition of the noise image data;

FIG. 15 is a graph showing an example of the relation between the numberof frames and the threshold for making a determination in step S608;

FIG. 16A is a diagram showing an overview of the update of a composingposition of the dust image data;

FIG. 16B is a diagram showing an overview of a fine correction of thecomposing position of the dust image data;

FIG. 17 is a diagram showing an overview of composing processing;

FIG. 18 is a diagram showing an overview of composing processing inwhich the sizes of the scratch image data and the noise image data afterthe cutout do not correspond to the size of image data to be composed;

FIG. 19 is a flowchart showing shading processing;

FIG. 20A is a diagram showing an example of a shading effect which isapplied when a magnification factor a is 0.5;

FIG. 20B is a diagram showing an example of a shading effect which isapplied when the magnification factor a is 1.5;

FIG. 21 is a flowchart showing processing to apply a granular noiseeffect;

FIG. 22 is a graph showing the relation between Wn, Hn, Wi, and Hi;

FIG. 23 is a flowchart showing pseudo-random number acquiringprocessing;

FIG. 24 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of RAW data to perform the special imageprocessing;

FIG. 25 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of RAW data demagnified by a RAWresizing unit to perform the special image processing;

FIG. 26 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of a random number generated duringphotography to perform the special image processing;

FIG. 27 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of YC data obtained by subjecting RAWdata to the basic image processing to perform the special imageprocessing;

FIG. 28 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of various conditions during photographyto perform the special image processing;

FIG. 29A and FIG. 29B are diagrams showing file structures of imagefiles when information for generating a random seed is recorded in animage file;

FIG. 30 is a flowchart showing reproduction processing; and

FIG. 31 is a flowchart showing editing processing.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the drawings. FIG. 1 is a block diagram showing theconfiguration of a digital camera as an example of an imaging apparatuscomprising an image processing apparatus according to one embodiment ofthe present invention. A digital camera 1 shown in FIG. 1 is a digitalcamera having an interchangeable lens. However, the digital camera 1does not necessarily have to be a digital camera having aninterchangeable lens, and may be a digital camera having an integrallens. The technique according to the present embodiment is alsoapplicable to, for example, a mobile telephone having an imagingfunction, or a portable terminal having an imaging function.

The digital camera 1 shown in FIG. 1 includes an interchangeable lens100 and a camera body 200. The interchangeable lens 100 is configured tobe removable from the camera body 200. When the interchangeable lens 100is attached to the camera body 200, the interchangeable lens 100 isconnected to the camera body 200 in communication with each other. As aresult, the interchangeable lens 100 is operable under the control ofthe camera body 200.

The interchangeable lens 100 includes a lens 102, a diaphragm 104, adriver 106, a microcomputer 108, and a flash memory 110.

The lens 102 is an optical system for collecting a light flux from anunshown subject to an image pickup device 204 in the camera body 200.The lens 102 has lenses such as a focus lens and a zoom lens. Thediaphragm 104 is configured to open and shut, and adjusts the amount ofthe light flux which has entered via the lens 102. The driver 106 has amotor and such like. Under the control of the microcomputer 108, thedriver 106 drives the focus lens and the zoom lens in the lens 102 inits optical axis direction, and drives the diaphragm 104 to open andshut.

The microcomputer 108 is connected to a microcomputer 234 in the camerabody 200 via an interface (I/F) 112 in communication with each otherwhen the interchangeable lens 100 is connected to the camera body 200.This microcomputer 108 drives the driver 106 under the control of themicrocomputer 234. The microcomputer 108 communicates lens informationregarding the interchangeable lens 100 stored in the flash memory 110 tothe microcomputer 234 via the I/F 112.

Lens information such as aberration information regarding the lens 102,and programs necessary to execute the operation of the interchangeablelens 100 are stored in the flash memory 110.

The camera body 200 has a mechanical shutter 202, the image pickupdevice 204, an analog processing unit 206, an analog/digital (AD)converting unit 208, a RAW resizing unit 210, a bus 212, an SDRAM 214,an AE processing unit 216, an AF processing unit 218, an imageprocessing unit 220, a subject detecting unit 222, a display driver 224,a display unit 226, an image compressing/decompressing unit 228, amemory interface (I/F) 230, a recording medium 232, the microcomputer234, an operating unit 236, and a flash memory 238.

The mechanical shutter 202 is configured to move a photoelectricconversion surface of the image pickup device 204 into a shaded state oran exposed state. The exposure time of the image pickup device 204 isadjusted by the movement of the mechanical shutter 202.

The image pickup device 204 has the photoelectric conversion surface onwhich the light flux from the subject collected via the lens 102 isformed into an image. The photoelectric conversion surface is configuredby two-dimensionally arrayed pixels. A color filter is provided on thelight entrance side of the photoelectric conversion surface. This imagepickup device 204 converts a figure (subject figure) corresponding tothe light flux formed on the photoelectric conversion surface to anelectric signal (hereinafter referred to as an image signal)corresponding to the light amount, and then outputs the electric signal.Here, image pickup devices having various configurations of, forexample, a CCD type and CMOS type are known as the image pickup device204. Various arrangements such as a Bayer arrangement are known as colorarrangements of the color filter. In the present embodiment, theconfiguration of the image pickup device 204 is not limited to aparticular configuration, and image pickup devices having variousconfigurations can be used. The image pickup device 204 may have anelectronic shutter function to electronically control the exposure time.In the following explanation, the image pickup device 204 has theelectronic shutter function.

The analog processing unit 206 subjects the image signal obtained by theimage pickup device 204 to analog processing such as correlated doublesampling (CDS) processing and automatic gain control (AGC) processing.The AD converting unit 208 converts the image signal analog-processed inthe analog processing unit 206 to a digital signal (hereinafter referredto as RAW data). Here, the RAW data is “raw” image data before beingsubjected to image processing in the image processing unit 220.

Here, the image pickup device 204, the analog processing unit 206, andthe AD converting unit 208 function as an imaging unit.

The RAW resizing unit 210 resizes the RAW data obtained in the ADconverting unit 208. The resizing is performed by interpolation. Whendemagnifying processing is performed as the resizing processing, forexample, processing which uses the average value of adjacent pixels asdata regarding the pixels after the demagnification is performed foreach of the pixels constituting the RAW data. When magnifying processingis performed as the resizing processing, for example, processing whichinserts the pixel having the average value of adjacent pixels into theadjacent pixels is performed for each of the pixels constituting the RAWdata. As shown in FIG. 1, the AD converting unit 208 is connected to thebus 212 via the RAW resizing unit 210, and is also connected to the bus212 without passing through the RAW resizing unit 210. Therefore, in thepresent embodiment, both the RAW data after resized in the RAW resizingunit 210 and the RAW data before resized in the RAW resizing unit 210can be acquired. The RAW resizing unit 210 may be configured to resizethe RAW data stored in the SDRAM 214 instead of resizing the RAW dataoutput from the AD converting unit 208. In addition to the methodsdescribed above, various interpolation methods such as nearest neighborinterpolation and linear interpolation are applicable to theinterpolation processing performed in the RAW resizing unit 210.

The bus 212 is a transfer channel for transferring various datagenerated inside the camera body 200. The SDRAM 214 is a storage unitfor temporarily storing various data generated inside the camera body200. This SDRAM 214 is also used as a buffer memory for image processingin the image processing unit 220.

The AE processing unit 216 calculates subject luminance by using imagedata (e.g., the RAW data). The AF processing unit 218 extracts signalsof a high-frequency component from the image data (e.g., the RAW data),and adds up the extracted signals of the high-frequency component toacquire an AF evaluation value.

The image processing unit 220 performs various kinds of image processingfor the RAW data. Here, the image processing performed in the imageprocessing unit 220 is image processing such that the finish of theimage data and an effect thereon will be a predetermined finish andeffect. The finish here refers to, for example, an appearance and astyle during display. The effect refers to, for example, an effect whichprovides a predetermined impression to a user during display. This imageprocessing unit 220 has a basic image processing unit 2201 and a specialimage processing unit 2202.

The basic image processing unit 2201 subjects the image data to basicimage processing necessary to display or record images. This basic imageprocessing includes, for example, optical black (OB) subtractionprocessing, white balance (WB) correction processing, synchronizationprocessing, color reproduction processing, luminance changingprocessing, edge enhancement processing, and noise reduction processing.The optical black subtraction processing is processing for subtractingand removing a dark current component (optical black) of the RAW data.The white balance correction processing is processing for amplifyingeach color component of the RAW data in a predetermined gain amount tocorrect the color balance of the image. The synchronization processingis processing for converting image data in which one pixel correspondsto one color component, such as the raw data output via the image pickupdevice 204 in accordance with the Bayer arrangement, to RGB data inwhich one pixel corresponds to more than one color component. The colorreproduction processing includes various kinds of processing such thatthe color reproduction of an image will be appropriate. This processingis, for example, color matrix calculation processing. This color matrixcalculation processing is processing for multiplying the RGB data by,for example, a color matrix coefficient corresponding to a white balancemode. In addition, the corrections of saturation and hue are also madeas the color reproduction processing. The luminance changing processingis processing for converting the RGB data to YC (luminance and colordifference) data, and changing the luminance characteristics of Y dataso that the luminance characteristics will be suitable to display andrecording. The luminance characteristics of the RGB data may be changedas the luminance changing processing. The edge enhancement processing isprocessing for multiplying, by an edge enhancement coefficient, an edgesignal extracted from the image data (the RAW data, the RGB data, or theYC data) by use of, for example, a band pass filter, and adding theresult to the original image data to enhance an edge (outline) componentin the image data. The noise reduction processing is processing forremoving a noise component in the image data (the RGB data or the YCdata) by, for example, coring processing.

The special image processing unit 2202 subjects the image data (the RGBdata or the YC data) to special image processing to provide a specialvisual effect. The special image processing unit 2202 in the presentembodiment performs processing to apply at least a noise effect as thespecial image processing. The noise effect is an effect to applypredetermined noise to the image to provide a predetermined impression(e.g., an impression of an image as if it has been obtained by filmphotography) to the user. To perform the processing to apply the noiseeffect, the special image processing unit 2202 includes a random seedgenerating unit 2202 a, a pseudo-random number generating unit 2202 b, acutout position calculating unit 2202 c, and a composing unit 2202 d.The random seed generating unit 2202 a generates a random seed forinitializing a pseudo-random number sequence. The pseudo-random numbergenerating unit 2202 b has a pseudo-random number generator, andgenerates a pseudo-random number sequence in accordance with the randomseed generated in the random seed generating unit 2202 a. Here, thepseudo-random number sequence is a sequence having pseudo-randomness,and is a sequence which is characterized in that the same sequence isgenerated from the same random seed. A linear congruential method, anXOR shift method, and a Mersenne Twister method are known as methods ofgenerating the pseudo-random numbers. In the present embodiment, thegenerating method is not particularly limited as long as a pseudo-randomnumber sequence can be generated. The cutout position calculating unit2202 c calculates cutout positions of scratch image data and noise imagedata which are image data necessary to apply the noise effect by usingthe pseudo-random number sequence generated in the pseudo-random numbergenerating unit 2202 b. The scratch image data and the noise image datawill be described in detail later. The composing unit 2202 d cuts outparts of the scratch image data and the noise image data in accordancewith the cutout positions calculated in the cutout position calculatingunit 2202 c, magnifies the scratch image data and the noise image datathat have been cut out as needed, and then composes (superimposes) thedata on the image data to which the noise effect is to be applied. Thecomposing unit 2202 d also composes (superimposes) dust image data usedas needed to apply the noise effect on the image data to which the noiseeffect is to be applied.

The subject detecting unit 222 detects a subject (e.g., a human face) inthe image data (e.g., YC data). When the subject is a face, the face canbe detected by a known face detection technique such as templatematching. Even a subject other than the face can be detected by a knownmethod such as the template matching or characteristic amount detection.

The display driver 224 resizes, in accordance with the display size ofthe display unit 226, the image data obtained in the image processingunit 220 or an image data obtained by decompression in the imagecompressing/decompressing unit 228, and converts the resized image datato a video signal, and then outputs the video signal to the display unit226. The display unit 226 is, for example, a liquid crystal display(LCD). The display unit 226 displays an image based on the video signalinput from the display driver 224.

In the recording of the image, the image compressing/decompressing unit228 subjects the image data obtained by the image processing in theimage processing unit 220 to still image compressing processing in aJPEG format or TIFF format or to moving image compressing processing inan MPEG format or an H.264 format. The image compressing/decompressingunit 228 decompresses the compressed image data during the reproductionof the image.

The I/F 230 is an interface for the microcomputer 234 and such like toaccess the recording medium 232. The recording medium 232 is, forexample, a memory card removable from the camera body 200. Image files,for example, are recorded in the recording medium 232. The image file isa file in which header information is added to the image data compressedby the image compressing/decompressing unit 228. The recording medium232 may be fixed to the camera body 200 (may be unremovable).

The microcomputer 234 has overall control of the operation of each ofthe components of the camera body 200 including the mechanical shutter202, the image pickup device 204, and the display driver 224. Themicrocomputer 234 also performs AE processing using the subjectluminance calculated in the AE processing unit 216, and AF processingusing the AF evaluation value calculated in the AF processing unit 218.Moreover, the microcomputer 234 also controls the operation of theinterchangeable lens 100 when the interchangeable lens 100 is attached.

The operating unit 236 includes various operational components to beoperated by the user. For example, the operating unit 236 in the presentembodiment has, as the operational components, a release button, amoving image button, a menu button, a reproduction button, and a powerbutton. The release button has a two-step switch for a first (1st)release switch and a second (2nd) release switch. When the releasebutton is pressed halfway by the user and the first release switch isturned on accordingly, the microcomputer 234 performs photographicpreparation processing such as AE processing and AF processing. When therelease button is fully pressed and the second release switch is turnedon accordingly, the microcomputer 234 performs still image recordingprocessing. The moving image button instructs the microcomputer 234 toperform moving image photography. When the moving image button ispressed, the microcomputer 234 performs moving image recordingprocessing. When the moving image button is pressed during the executionof the moving image recording processing, the microcomputer 234 finishesthe moving image recording processing. The menu button is an operationunit for instructing to display a menu screen. On the menu screen, theuser can change various settings of the camera body 200. In the presentembodiment, the user sets, for example, a special image processing modeon the menu screen. In accordance with this special image processingmode, the contents of the special image processing applied in thespecial image processing unit 2202 are set. The reproduction button isan operation unit for instructing the microcomputer 234 to reproduce astill image file or a moving image file. The power button is anoperation unit for instructing to turn on or off the camera body 200.Here, functions equivalent to the release button, the moving imagebutton, the menu button, and the reproduction button described above maybe provided by a touch panel. That is, there may be no physicaloperational components such as the buttons.

Various parameters necessary for the operation of the camera body 200are stored in the flash memory 238, such as parameters necessary for theoperation of the image processing unit 220: a white balance gain for awhite balance correction, color matrix coefficient for a color matrixcalculation, and various functions (gamma functions) for changing theluminance. Here, the scratch image data, the noise image data, and thedust image data are stored in the flash memory 238 according to thepresent embodiment as the parameters necessary for the special imageprocessing in the image processing unit 220. Various programs to beexecuted by the microcomputer 234 are also stored in the flash memory238.

The operation of the above digital camera is described below. FIGS. 2Aand 2B are flowcharts showing the main operation of the digital cameraaccording to the present embodiment. The operation in FIGS. 2A and 2B isperformed, for example, when the power of the digital camera 1 is turnedon. After the power is turned on, the microcomputer 234 performsinitialization processing (step S101). In the initialization processing,the microcomputer 234 performs processing to turn off a recording flagset in its register. The recording flag is a flag that indicates whethermoving images are being recorded. The recording flag that is turned offindicates that moving images are not being recorded. On the other hand,the recording flag that is turned on indicates that moving images arebeing recorded.

The microcomputer 234 then determines whether the reproduction button ofthe operating unit 236 has been pressed by the user (step S102). When itis determined in step S102 that the reproduction button has beenpressed, the microcomputer 234 performs reproduction processing (stepS103). The reproduction processing will be described in detail later.

When it is determined in step S102 that the reproduction button has notbeen pressed, the microcomputer 234 determines whether to perform camerasetting (step S104). For example, when the menu button of the operatingunit 236 is pressed by the user, the microcomputer 234 determines toperform the camera setting. When it is determined in step S104 toperform the camera setting, the microcomputer 234 controls the displaydriver 224 to cause the display unit 226 to display the menu screen, andthen performs camera setting processing (step S105). In the camerasetting processing, the microcomputer 234 waits for an instruction fromthe user to change the camera settings. When instructed to make somechanges to the camera settings, the microcomputer 234 changes theappropriate camera setting. In this camera setting processing, changesare made in the settings regarding the finish of the image; for example,the setting of the recording format of the image during still imagephotography or moving image photography, the white balance (WB) mode,contrast setting, the setting of the degree of edge enhancement(sharpness), and the setting of luminance characteristic changes (gammasetting). In addition, the special image processing mode, for example,the effect may also be set in the camera setting processing.

When it is determined in step S104 not to perform the camera setting,the microcomputer 234 determines whether the moving image button of theoperating unit 236 has been pressed by the user (step S106). When it isdetermined in step S106 that the moving image button has been pressed,the microcomputer 234 toggles the recording flag (step S107). That is,the microcomputer 234 turns on the recording flag that is off, and turnsoff the recording flag that is on. The microcomputer 234 then determineswhether moving images are being recorded at present, that is, whetherthe recording flag is on (step S108).

When it is determined in step S108 that the recording flag is on, themicrocomputer 234 creates a moving image file, and prepares for movingimage data to be recorded (step S109). When it is determined in stepS108 that the recording flag is not on, the microcomputer 234 closes themoving image file (step S110).

In case of being determined in step S106 that the moving image buttonhas not been pressed, after being closed the moving image file in stepS110, or after the creation of the moving image file in step S109, themicrocomputer 234 performs effect setting processing (step S111). Theeffect setting processing will be described in detail later.

After the effect setting processing, the microcomputer 234 againdetermines whether moving images are being recorded at present, that is,whether the recording flag is on (step S112). When it is determined instep S112 that the recording flag is off, the microcomputer 234determines whether the release button of the operating unit 236 has beenpressed halfway by the user so that the state of the release button haschanged from the off-state to the on-state of the 1st release switch(step S113).

When it is determined in step S113 that the state of the release buttonhas changed to the on-state of the 1st release switch, the microcomputer234 performs the AE processing and the AF processing (step S114). In theAE processing, the microcomputer 234 causes the AE processing unit 216to calculate subject luminance. The microcomputer 234 then decides ashutter speed (Tv value), an aperture value (Av value), and the ISOduring still image photography in accordance with the subject luminancecalculated by the AE processing unit 216. Here, the shutter speed, theaperture value, and the ISO may be decided so that the luminance of thesubject detected by the subject detecting unit 222 will be proper. Inthe AF processing, the microcomputer 234 causes the AF processing unit218 to acquire an AF evaluation value. The microcomputer 234 thenevaluates contrast by the AF evaluation value acquired by the AFprocessing unit 218, and at the same time instructs the microcomputer108 to drive the focus lens of the lens 102 by a slight amount. Themicrocomputer 234 then instructs the microcomputer 108 to stop thedriving of the focus lens at the point where the contrast is maximized.This AF processing is what is known as contrast-type AF processing.Phase difference AF processing may also be used as the AF processing.The subject detected by the subject detecting unit 222 may be focused.

After the AE processing and the AF processing, the microcomputer 234determines whether the power of the digital camera 1 has been turned off(step S115). When it is determined in step S115 that the power of thedigital camera 1 has not been turned off, the microcomputer 234 performsprocessing in and after step S102. On the other hand, when it isdetermined in step S115 that the power of the digital camera has beenturned off, the microcomputer 234 finishes the processing in FIGS. 2Aand 2B.

When it is determined in step S113 that the state of the release buttonhas not changed to the on-state of the 1st release switch, themicrocomputer 234 determines whether the release button of the operatingunit 236 has been fully pressed by the user so that the state of therelease button has changed to the on-state of the 2nd release switch(step S116).

When it is determined in step S116 that the state of the release buttonis the on-state of the 2nd release switch, the microcomputer 234performs photography processing using the mechanical shutter 202 (stepS117). Accordingly, the microcomputer 234 sets a gain control amount(amplification factor) in the analog processing unit 206 in accordancewith the ISO decided in the AE processing, and sends an F-value decidedin the AE processing to the microcomputer 108. The microcomputer 234then actuates the mechanical shutter 202 in accordance with the exposuretime decided in the AE processing to control the exposure of the imagepickup device 204 synchronously with the driving of the diaphragm 104controlled by the microcomputer 108. The RAW data is stored in the SDRAM214 by this photography processing.

After having performed the photography processing using the mechanicalshutter 202, the microcomputer 234 causes the image processing unit 220to perform image processing for the RAW data which has been stored inthe SDRAM 214 by the photography processing (step S118). The imageprocessing will be described in detail later.

After the image processing, the microcomputer 234 performs processing torecord the image data stored as the result of the image processing inthe SDRAM 214, as a still image file in a set still image recordingformat (step S119). At the same time, the microcomputer 234 inputs theimage data stored in the SDRAM 214 to the imagecompressing/decompressing unit 228 to instruct the imagecompressing/decompressing unit 228 to perform still image compressingprocessing. In response to this instruction, the imagecompressing/decompressing unit 228 performs the still image compressingprocessing in accordance with the preset recording mode, and stores thecompressed image data in the SDRAM 214. The microcomputer 234 then readsthe image data compressed by the image compressing/decompressing unit228 from the SCRAM 214, creates a still image file from the read imagedata, and records the created still image file in the recording medium232.

When it is determined in step S116 that the state of the release buttonis not the on-state of the 2nd release switch, the microcomputer 234performs the AF processing (step S120). This AE processing is processingfor moving image photography or live view display. After the AEprocessing, the microcomputer 234 performs photography processing usingthe electronic shutter (step S121). In this photography processing, themicrocomputer 234 actuates the electronic shutter function of the imagepickup device 204 in accordance with the exposure time decided in the AEprocessing to control the exposure of the image pickup device 204. TheRAW data is stored in the SDRAM 214 by this photography processing.

After having performed the photography processing using the electronicshutter, the microcomputer 234 causes the image processing unit 220 toperform image processing for the RAW data which has been stored in theSDRAM 214 by the photography processing (step S122). The imageprocessing will be described in detail later.

After the image processing, the microcomputer 234 performs the live viewdisplay (step S123). In the live view display, the microcomputer 234inputs the image data stored in the SDRAM 214 to the display driver 224as the result of the image processing. Accordingly, the display driver224 converts the input image data to a video signal, and then outputsthe video signal to the display unit 226. The display unit 226 displaysan image based on this video signal. This live view display allows theuser to, for example, check the composition using the display unit 226.

After the live view display, the microcomputer 234 determines whethermoving images are being recorded at present, that is, the recording flagis on (step S124). When it is determined in step S124 that the recordingflag is on, the microcomputer 234 skips the processing in step S125.When it is determined in step S124 that the recording flag is on, themicrocomputer 234 performs processing to record the image data stored asthe result of the image processing in the SDRAM 214, as a moving imagefile in a set moving image recording format (step S125). At the sametime, the microcomputer 234 inputs the moving image data stored in theSDRAM 214 to the image compressing/decompressing unit 228 to instructthe image compressing/decompressing unit 228 to perform moving imagecompressing processing. In response to this instruction, the imagecompressing/decompressing unit 228 performs the moving image compressingprocessing in accordance with the preset recording mode, and stores thecompressed image data in the SDRAM 214. The microcomputer 234 then readsthe moving image data compressed by the image compressing/decompressingunit 228 from the SDRAM 214, and additionally records the read movingimage data in the previously created moving image file. When therecording of the moving image data has finished, information such as thenumber of frames is recorded in a header recording portion of the movingimage file.

FIG. 3 is a flowchart showing the effect setting processing. In theeffect setting processing, the user sets the contents of an effectapplied to the image (still image or moving image or live view). Inaccordance with the setting of this effect, the effect is applied to theimage during the special image processing described later.

In FIG. 3, the microcomputer 234 determines whether the user hasinstructed to apply a film noise effect to the image (step S201). Theuser instructs to apply the film noise effect on a menu screen similarto that in, for example, the camera setting processing. When it isdetermined in step S201 that the user has not instructed to apply thefilm noise effect, the microcomputer 234 skips the processing in stepS202. When it is determined in step S201 that the user has instructed toapply the film noise effect, the microcomputer 234 sets the imageprocessing unit 220 (special image processing unit 2202) to apply thefilm noise effect (step S202).

The microcomputer 234 then determines whether the user has instructed toapply a shading effect to the image (step S203). When it is determinedin step S203 that the user has not instructed to apply the shadingeffect, the microcomputer 234 skips the processing in step S204. When itis determined in step S203 that the user has instructed to apply theshading effect, the microcomputer 234 sets the image processing unit 220(special image processing unit 2202) to apply the shading effect duringthe special image processing (step S204).

The microcomputer 234 then determines whether the user has instructed toapply a granular noise effect to the image (step S205). When it isdetermined in step S205 that the user has not instructed to apply thegranular noise effect, the microcomputer 234 skips the processing instep S206. When it is determined in step S205 that the user hasinstructed to apply the granular noise effect, the microcomputer 234sets the image processing unit 220 (special image processing unit 2202)to apply the granular noise effect during the special image processing(step S206). The microcomputer 234 then finishes the processing in FIG.3.

FIG. 4 is a flowchart showing the image processing. When the imageprocessing is started, the basic image processing unit 2201 subjects theRAW data stored in the SDRAM 214 to the basic image processing (stepS301). The special image processing unit 2202 then subjects the imagedata (YC data) stored in the SDRAM 214 as the result of the basic imageprocessing to the special image processing (step S302). Thus, the imageprocessing is finished. The basic image processing and the special imageprocessing will be described in detail below.

FIG. 5 is a flowchart showing the basic image processing. After thestart of the basic image processing, the basic image processing unit2201 performs OB subtraction processing (step S401). In the OBsubtraction processing, the basic image processing unit 2201 subtractsan optical black (OB) value from the input RAW data to remove the darkcurrent component in the RAW data.

After the OB subtraction processing, the basic image processing unit2201 performs the WB correction processing (step S402). In the WBcorrection processing, the basic image processing unit 2201 multipliesthe RAW data subjected to the OB subtraction processing by a WB gaincorresponding to a WB mode preset by the user, and thereby corrects thecolor balance of the image. When the user has set an automatic WB mode,the basic image processing unit 2201 analyzes the photographed RAW data,and then multiplies the RAW data by a WB gain corresponding to anestimated light source.

After the WB correction processing, the basic image processing unit 2201performs the synchronization processing when the format of the RAW datais the Bayer arrangement (step S403). In the synchronization processing,the basic image processing unit 2201 uses the interpolation processingto synchronize the WB-corrected RAW data. In this way, the RAW data inwhich one pixel has one color component of RGB is converted to RGB datain which one pixel has three color components of RGB.

After the synchronization processing, the basic image processing unit2201 performs the color reproduction processing (step S404). In thecolor reproduction processing, the basic image processing unit 2201multiplies each pixel of the RGB data by the color matrix coefficientcorresponding to the set. WB mode, and thereby performs the colorconversion of the RGB data. Further, the basic image processing unit2201 corrects the color so that the hue and saturation of thecolor-converted RGB data will be appropriate, thereby adjusting thecolor reproduction of the image.

After the color reproduction processing, the basic image processing unit2201 performs the luminance changing processing (step S405). In theluminance changing processing, the basic image processing unit 2201gamma-converts the RGB data subjected to the color reproductionprocessing, and further converts the gamma-converted RGB data to YC(luminance and color difference) data, and then gamma-converts the Ydata. Only one of the RGB and the Y data may be gamma-converted.

After the luminance changing processing, the basic image processing unit2201 performs the edge enhancement processing (step S406). In the edgeenhancement processing, the basic image processing unit 2201 subjectsthe Y data after the luminance changing processing to band pass filterprocessing to extract an edge signal, and multiplies the extracted edgesignal by a coefficient corresponding to an edge enhancement amount. Thebasic image processing unit 2201 then adds the edge component multipliedby the coefficient to the original Y data to enhance the edge componentin the image.

After the edge enhancement processing, the basic image processing unit2201 performs the noise reduction (NR) processing (step S407). The basicimage processing unit 2201 then finishes the processing in FIG. 5. Inthe noise reduction processing, the basic image processing unit 2201frequency-separates the Y data subjected to the edge enhancementprocessing, and reduces the noise component in the image by, forexample, coring processing in accordance with the frequency. The noisecomponent may be reduced in Cb data or Cr data. The data after the noisereduction processing is again converted to the RGB format by apredetermined matrix operation when the recording format is the TIFFformat.

FIG. 6 is a flowchart showing the special image processing. FIG. 6 showsan example in which the processing to apply the film noise effect, theprocessing to apply the shading effect, and the processing to apply thegranular noise effect are performed as the special image processing. Inthe special image processing, the special image processing other thanthe processing shown in FIG. 6, for example, blurring processing may beadditionally performed.

In FIG. 6, the special image processing unit 2202 determines whether thefilm noise effect is set to be applied (step S501). When it isdetermined in step S501 that the film noise effect is not set to beapplied, the special image processing unit 2202 skips the processing instep S502. When it is determined in step S501 that the film noise effectis set to be applied, the special image processing unit 2202 performsprocessing to apply the film noise effect to the image data (YC data)(step S502). This processing will be described in detail later.

The special image processing unit 2202 then determines whether theshading effect is set to be applied (step S503). When it is determinedin step S503 that the shading effect is not set to be applied, thespecial image processing unit 2202 skips the processing in step S502.When it is determined in step S503 that the shading effect is set to beapplied, the special image processing unit 2202 performs processing toapply the shading effect to the image data (YC data) (step S504). Thisprocessing will be described in detail later.

The special image processing unit 2202 then determines whether thegranular noise effect is set to be applied (step S505). When it isdetermined in step S505 that the granular noise effect is not set to beapplied, the special image processing unit 2202 skips the processing instep S506 and then finishes the processing in FIG. 6. When it isdetermined in step S505 that the granular noise effect is set to beapplied, the special image processing unit 2202 performs processing toapply the granular noise effect to the image data (YC data) (step S506).The special image processing unit 2202 then finishes the processing inFIG. 6. This processing to apply the granular noise effect will bedescribed in detail later.

Now, the processing to apply the film noise effect is described. Beforethe detailed description of the processing to apply the film noiseeffect, the scratch image data, the noise image data, and the dust imagedata are described. FIG. 7 is a diagram showing the scratch image data.FIG. 8 is a diagram showing the noise image data. FIG. 9 is a diagramshowing the dust image data. The scratch image data, the noise imagedata, and the dust image data are image data in which a specific noisethat can be generated during film photography is patterned. The sizes ofthe scratch image data, the noise image data, and the dust image dataare decided based on the image data of a predetermined size (e.g.,1980×1080 pixels).

As shown in FIG. 7, the scratch image data is composed of randomlyarranged longitudinal streak patterns different in length. When thescratch image data having the arrangement of longitudinal streakpatterns is superimposed on the image data (YC data), noise resultingfrom scratches produced when the film is longitudinally moved isreproduced on the image data. Here, lateral streak patterns may bearranged instead of the patterns in FIG. 7. When the scratch image datahaving the arrangement of lateral streak patterns is superimposed on theimage data (YC data), noise resulting from scratches produced when thefilm is laterally moved is reproduced on the image data.

The scratch image data is image data having a high correlation in adirection (longitudinal direction in FIG. 7) along the streak patterns.Thus, the scratch image data is demagnified (e.g., to about ⅛ to ⅙ whenthe size of the scratch image data is decided based on the image datahaving 1980×1080 pixels) in a direction of the high correlation(longitudinal direction in FIG. 7), and scratch image data obtained bythe magnification of this demagnified scratch image data is highlycorrelated with the original scratch image data which is notdemagnified. That is, even the scratch image data that is demagnifiedcan be used without any problem if it is magnified later. Thus, it ispreferable that the demagnified scratch image data is stored in theflash memory 238. If the demagnified scratch image data is stored in theflash memory 238, the capacity of the flash memory 238 can be saved. Ifprocessing is performed by use of the demagnified scratch image databefore actual superposition, the band of the SDRAM 214 can also besaved.

As shown in FIG. 8, the noise image data is composed of randomly andtwo-dimensionally arranged granular patterns. When the noise image datais superimposed on the image data (YC data), noise resulting from, forexample, dust produced at a film developing stage in film photography isreproduced on the image data. The granular patterns in the noise imagedata are randomly arranged in both the longitudinal and lateraldirections. Therefore, in contrast to the scratch image data, the noiseimage data is not image data that is correlated in a particulardirection. Thus, it is preferable that the noise image data that is notdemagnified is stored in the flash memory 238. However, when the noiseimage data needs to be demagnified because of the relation with thecapacity of the flash memory 238 and the band of the SDRAM 214, it ispreferable to demagnify the noise image data at the same magnificationfactor in the longitudinal direction and the lateral direction.

As shown in FIG. 9, the dust image data is image data in which noisecaused by filiform dust is patterned. In the example of FIG. 9, the dustimage data includes five patterns of image data: image data A to Dregarding four different patterns of dust, and image data E withoutdust. One of the five patterns of image data is randomly selected andsuperimposed on the image data (YC data). When the dust image data issuperimposed on the image data (YC data), noise resulting from dustadhering to the film surface during film photography is reproduced onthe image data. Here, FIG. 9 shows the example in which the dust imagedata has five patterns of image data. The number of patterns stored asthe dust image data is not exclusively five.

FIG. 10 is a flowchart showing processing to apply the film noiseeffect. In FIG. 10, the special image processing unit 2202 causes therandom seed generating unit 2202 a and the pseudo-random numbergenerating unit 2202 b to acquire a pseudo-random number necessary toapply the film noise (step S601). In the processing shown in FIG. 10,ten pseudo-random numbers (R[0] to R[9]) are acquired by way of example.Here, the pseudo-random numbers R[0] to R[9] have values ranging from 0to a random number maximum value MAX. The specific way to acquire thepseudo-random numbers R[0] to R[9] will be described later.

After having acquired the pseudo-random numbers R[0] to R[9], thespecial image processing unit 2202 determines whether to greatly(randomly) update the cutout position of the scratch image data to besuperposed on the image data (YC data) (step S602). Here, the cutoutposition is a referential position to decide a cutout range of thescratch image data, and corresponds to upper left coordinates of thescratch image data. In the present embodiment, the cutout range of thescratch image data is randomly updated at the time of the superpositionof the scratch image data to reproduce the randomness of the noiseresulting from the scratches. In step S602, when the image data to whichthe film noise effect is to be applied is still image data, thedetermination is always Yes. When the image data to which the film noiseeffect is to be applied is not still image data (e.g., the image data ismoving image data or image data for the live view display), thedetermination corresponding to the initial frame is Yes, and thedeterminations corresponding to the subsequent frames are randomly Yes.For example, when the pseudo-random number R[0] is equal to or higherthan the threshold which varies depending on the number of frames afterthe update of the cutout position, the determination is Yes.

FIG. 11 is a graph showing an example of the relation of the thresholdused in the determination in step S602 with the number of frames. Thehorizontal axis in FIG. 11 indicates the number of lapsed frames whenthe frame in which the cutout position has been updated is 0. Thevertical axis in FIG. 11 indicates the value of the threshold. In theexample of FIG. 11, the threshold is higher than the maximum value ofthe pseudo-random number R[0] between the 0th frame and the 14th frame.Therefore, the determination is always No in step S602 up to the 14thframe after the update. From the 15th frame to 29th frame, whether thedetermination is Yes or No depends on the value of the pseudo-randomnumber R[0]. That is, from the 15th frame to 29th frame after theupdate, whether to update the cutout position is randomly decided. Inthe 30th frame, the threshold reaches the maximum value (0 in theexample shown) of the pseudo-random number R[0]. Therefore, when thereis no update in 30 frames after the update, the determination is alwaysYes in step S602. When the scratch is flowing streak noise, the user isless uncomfortable. Accordingly, as shown in FIG. 11, while the cutoutposition of the scratch image data is not updated frequently, the cutoutposition is changed at times. In this way, noise similar to the noiseresulting from the actual scratch is reproduced on the image data. Here,the relation in FIG. 11 is illustrative only and can be suitablychanged.

When it is determined in step S602 to update the cutout position, thespecial image processing unit 2202 causes the cutout positioncalculating unit 2202 c to update the cutout position of the scratchimage data (step S603). For example, the X coordinates on the upper leftside of the cutout position are updated to the position indicated by thepseudo-random number R[1], and the Y coordinates on the upper left sideof the cutout position are updated to the position indicated by thepseudo-random number R[2]. Here, the minimum value of the pseudo-randomnumber R[1] corresponds to, for example, the coordinates on the upperleft side of the cutout range in which the left end of the cutout rangecontacts the left end of the scratch image data, and the maximum valueof the pseudo-random number R[1] corresponds to, for example, thecoordinates on the upper left side of the cutout range in which theright end of the cutout range contacts the right end of the scratchimage data. The minimum value of the pseudo-random number R[2]corresponds to, for example, the coordinates on the upper left side ofthe cutout range in which the upper end of the cutout range contacts theupper end of the scratch image data, and the maximum value of thepseudo-random number R[2] corresponds to, for example, the coordinateson the upper left side of the cutout range in which the lower end of thecutout range contacts the lower end of the scratch image data. Moreover,regarding the sizes of the cutout range of the scratch image data, forexample, the longitudinal size is the demagnification factor of theimage data to be composed, and the lateral size is the same size as thatof the image data to be composed.

FIG. 12A shows an overview of the update of the cutout position. Adashed frame in FIG. 12A indicates the cutout range before update, and adashed-dotted frame in FIG. 12A indicates the cutout range after update.The cutout position is updated by updating the cutout position (Xp, Yp)before update to the cutout position (x, y) after update, as shown inFIG. 12A. x and y are provided by, for example, (Equation 1):x=R[1]÷MAX×(Xmax−Xmin)+Xminy=R[2]÷MAX×(Ymax−Ymin)+Ymin  (Equation 1)wherein Xmax indicates the maximum value in the lateral direction, Xminindicates the minimum value in the lateral direction, Ymax indicates themaximum value in the longitudinal direction, and Ymin indicates theminimum value in the longitudinal direction.

When it is determined in step S602 not to update the cutout position,the special image processing unit 2202 determines whether thepseudo-random number R[2] (i.e., the Y coordinates of the cutoutposition) is equal to or lower than the random number maximum valueMAX×0.9 (step S604). When it is determined in step S604 that thepseudo-random number R[2] is equal to or lower than the random numbermaximum value MAX×0.9, the special image processing unit 2202 finelycorrects the cutout position of the scratch image data (step S605). Thecutout position is finely corrected so that the distance between thecutout position before update and the cutout position after update doesnot increase, that is, corrected within the limited range near thecutout position before update, as shown in FIG. 12B. In this case, thecutout position is upwardly changed at regular intervals in thelongitudinal direction, and randomly changed in the lateral direction.In the longitudinal direction, the cutout position is corrected towardthe lower end as shown in FIG. 12C in the next correction when thecutout position has reached the upper end. The corrected cutout position(x, y) in the case of the above correction is provided by, for example,(Equation 2):x=Xp+((R[1]−MAX/2)/MAX)×Jy=Yp−K  (Equation 2)wherein J is a lateral basic movement speed (pixel/frame) of the streakpattern decided at the time of designing. For example, J is 5. K is alongitudinal movement speed (pixel/frame) of the streak pattern which isdecided at the time of designing. For example, K is 10. When x is lowerthan the minimum value in the lateral direction as a result of thecalculation in (Equation 2), x is clipped to the minimum value in thelateral direction. In contrast, when x is higher than the maximum valuein the lateral direction as a result of the calculation in (Equation 2),x is clipped to the maximum value in the lateral direction. When y isequal to or lower than the minimum value in the longitudinal directionas a result of the calculation in (Equation 2), that is, when the cutoutposition has reached the upper end, y is corrected to the maximum value(i.e., the lower end position) in the longitudinal direction in the nextframe. The X coordinates of the cutout position are randomly changed by(Equation 2) per frame within the range of five pixels (Xp±2.5 pixels).The Y coordinates of the cutout position are upwardly changed by tenpixels per frame. When it is determined in step S604 that thepseudo-random number R[2] is not equal to or lower than the randomnumber maximum value MAX×0.9, the special image processing unit 2202skips the processing in step S605 and then shifts the processing to stepS606. The condition for skipping the processing in step S605 may be athreshold other than the above-mentioned threshold. Alternatively, theprocessing in step S605 may not be skipped at all times.

The special image processing unit 2202 then determines whether to updatethe cutout position of the noise image data to be superposed on theimage data (YC data) (step S606). In the present embodiment, the cutoutrange of the noise image data is randomly updated at the time of thesuperposition of the noise image data to reproduce the randomness of thenoise resulting from, for example, dust. This determination of whetherto update the cutout position is the determination in step S606. In stepS606, when the image data to which the film noise effect is to beapplied is still image data, the determination is always Yes. When theimage data to which the film noise effect is to be applied is not stillimage data, the determination corresponding to the initial frame is Yes,and the determinations corresponding to the subsequent frames arerandomly Yes. For example, when the pseudo-random number R[3] is equalto or higher than the threshold which varies depending on the number offrames after the update of the cutout position, the determination isYes.

FIG. 13 is a graph showing an example of the relation between the numberof frames and the threshold for making a determination in step S606. Thehorizontal axis in FIG. 13 indicates the number of lapsed frames whenthe frame in which the cutout position has been updated is 0. Thevertical axis in FIG. 13 indicates the value of the threshold. In theexample of FIG. 13, the threshold falls between the 0-th frame and thepredetermined frame (the 7-th frame in the example shown). The thresholdis the minimum value (0 in the example shown) of the pseudo-randomnumber R[3] between the predetermined frame and the 10th frame.Therefore, the determination is always Yes in step S606 when there is noupdate for the predetermined frames after the update. In contrast toFIG. 11, there is no period in which the threshold is higher than therandom number maximum value in FIG. 13. Therefore, the cutout positionof the noise image data is frequently updated as compared to the scratchimage data. Here, the relation in FIG. 13 is illustrative only and canbe suitably changed.

When it is determined in step S606 not to update the cutout position,the special image processing unit 2202 skips the processing in stepS607. When it is determined in step S606 to update the cutout position,the special image processing unit 2202 causes the cutout positioncalculating unit 2202 c to update the cutout position of the noise imagedata (step S607). For example, the X coordinates on the upper left sideof the cutout position are updated to the position indicated by thepseudo-random number R[4], and the Y coordinates on the upper left sideof the cutout position are updated to the position indicated by thepseudo-random number R[5]. Here, the minimum value of the pseudo-randomnumber R[4] corresponds to, for example, the coordinates on the upperleft side of the cutout range in which the left end of the cutout rangecontacts the left end of the noise image data, and the maximum value ofthe pseudo-random number R[4] corresponds to, for example, thecoordinates on the upper left side of the cutout range in which theright end of the cutout range contacts the right end of the noise imagedata. The minimum value of the pseudo-random number R[5] corresponds to,for example, the coordinates on the upper left side of the cutout rangein which the upper end of the cutout range contacts the upper end of thenoise image data, and the maximum value of the pseudo-random number R[5]corresponds to, for example, the coordinates on the upper left side ofthe cutout range in which the lower end of the cutout range contacts thelower end of the noise image data. Moreover, the cutout range of thenoise image data is the same size as, for example, the image data to becomposed.

FIG. 14 shows an overview of the update of the cutout position. A dashedframe in FIG. 14 indicates the cutout range before update, and adashed-dotted frame in FIG. 14 indicates the cutout range after update.The cutout position of the noise image data is updated by updating thecutout position (Xp, Yp) before update to the cutout position (x, y)after update, as in the case of the scratch image data. x and y areprovided by, for example, (Equation 3):x=R[4]÷MAX×(Xmax−Xmin)+Xminy=R[5]÷MAX×(Ymax−Ymin)+Ymin  (Equation 3)

The special image processing unit 2202 then determines whether to updatethe dust image data to be superposed on the image data (YC data) (stepS608). In step S608, when the image data to which the film noise effectis to be applied is still image data, the determination is always Yes.When the image data to which the film noise effect is to be applied isnot still image data, the determination corresponding to the initialframe is Yes, and the determinations corresponding to the subsequentframes are randomly Yes. For example, when the pseudo-random number R[6]is equal to or higher than the threshold which varies depending on thenumber of frames after the update of the cutout position, thedetermination is Yes.

FIG. 15 is a graph showing an example of the relation between the numberof frames and the threshold for making a determination in step S608. Thehorizontal axis in FIG. 15 indicates the number of lapsed frames whenthe frame in which the cutout position has been updated is 0. Thevertical axis in FIG. 15 indicates the value of the threshold. In theexample of FIG. 15, the threshold linearly falls between the 0th frameand the 10th frame. In this case, the determination is always Yes instep S608 when there is no update before the 10th frame. There is noperiod in which the threshold is higher than the random number maximumvalue in FIG. 15 either. Therefore, the cutout position of the dustimage data is frequently changed as compared to the scratch image data.Here, the relation in FIG. 15 is illustrative only and can be suitablychanged.

When it is determined in step S608 to update the dust image data, thespecial image processing unit 2202 updates the dust image data (stepS609). For example, numbers are given to dust image data A to E shown inFIG. 9. In step S609, the currently selected dust image data is changedto the dust image data (e.g., A if the remainder is 0 when R[7] isdivided by 5, B if the remainder is 1, C if the remainder is 2, D if theremainder is 3, and E if the remainder is 4) indicated by thepseudo-random number R[7]. After the update of the dust image data, thespecial image processing unit 2202 changes the composing position of thedust image data (step S610). For example, the X coordinates on the upperleft side of the composing position are updated to the positionindicated by the pseudo-random number R[8], and the Y coordinates on theupper left side of the composing position are updated to the positionindicated by the pseudo-random number R[9]. Here, the minimum value ofthe pseudo-random number R[8] corresponds to, for example, thecoordinates of the left end of the image data to be composed, and themaximum value of the pseudo-random number R[8] corresponds to, forexample, the coordinates of the right end of the image data to becomposed. The minimum value of the pseudo-random number R[9] correspondsto, for example, the coordinates of the upper end of the image data tobe composed, and the maximum value of the pseudo-random number R[9]corresponds to, for example, the coordinates of the lower end of theimage data to be composed. FIG. 16A shows an overview of the update ofthe composing position of the dust image data. A dashed line in FIG. 16Aindicates the dust image data before update, and a dashed-dotted line inFIG. 16A indicates the dust image data after update. As shown in FIG.16A, not only the composing position but also the pattern of the dustimage data is changed in the case of the dust image data.

When it is determined in step S608 not to update the dust image data,the special image processing unit 2202 finely corrects the composingposition of the dust image data (step S611). The composing position isfinely corrected so that the distance between the composing positionbefore update and the composing position after update does not increase,as shown in FIG. 16B. In this case, the composing position is randomlychanged in both the longitudinal direction and the lateral direction.The corrected composing position (x, y) is provided by, for example,(Equation 4):x=Xp+((R[8]−MAX/2)/MAX)×Ly=Yp+HR[9]−MAX/2)/MAX)×M  (Equation 4)wherein L is a lateral basic movement speed (pixel/frame) of the dustimage data decided at the time of designing. For example, L is 5. M is alongitudinal movement speed (pixel/frame) of the dust image data decidedat the time of designing. For example, M is 5. Moreover, both x and yare values within the range of the lateral direction and thelongitudinal direction. Therefore, when x or y is lower than the minimumvalue in the lateral direction or the longitudinal direction as a resultof the calculation in (Equation 4), x or y is clipped to the minimumvalue in the lateral direction or the longitudinal direction. Incontrast, when x or y is higher than the maximum value in the lateraldirection or the longitudinal direction as a result of the calculationin (Equation 4), x or y is clipped to the maximum value in the lateraldirection or the longitudinal direction.

The special image processing unit 2202 then causes the composing unit2202 d to compose the scratch image data, the noise image data, and thedust image data with the image data to be composed (step S612). Thespecial image processing unit 2202 then finishes the processing in FIG.10. FIG. 17 is a diagram showing an overview of composing processing. Inthe composition, the size of the scratch image data is first adjusted tothe size of the image data to be composed. As described above, thescratch image data is demagnified in the longitudinal direction in whichthe correlation is high. Therefore, the scratch image data after thecutout is magnified in the longitudinal direction by the reciprocalnumber of the demagnification factor to adjust the size of the scratchimage data to the size of the image data to be composed. After the sizeof the scratch image data has been adjusted to the size of the imagedata to be composed, the scratch image data, the noise image data, andthe dust image data are composed. The three types of image data arecomposed; by multiplying by 0 if the value of each pixel is 0 (black),by multiplying by 1.0 if the value of each pixel is the maximum value(white), or by multiplying by a value of 0 to 1.0 depending on thebrightness if the value of each pixel is some other value. Aftercomposite noise image data has been obtained by the composition of thethree image data, the composite noise image data is composed with theimage data to be composed. This composition is also performed by, forexample, the multiplication of these image data. Here, the composingprocessing may be performed by some other method. For example, eachpixel may be compared, and the value of the darker pixel may be thecomposition result.

The sizes of the scratch image data, the noise image data, and the dustimage data are decided based on the image data of a predetermined size(e.g., 1980×1080 pixels). Therefore, it is preferable to magnify ordemagnify and then compose the scratch image data, the noise image data,and the dust image data depending on the size of the image data to becomposed. FIG. 18 is a diagram showing an overview of this composingprocessing. For example, when the size of the image data to be composedis 4000×3000 pixels (e.g., still image data), the noise image data andthe dust image data are magnified 4000/1920 times in the longitudinaldirection and the lateral direction and then composed. The scratch imagedata has been demagnified in the longitudinal direction. Therefore, thescratch image data is magnified (1/demagnification factor)×4000/1920times and then composed. Thus, the scratch image data, the noise imagedata, and the dust image data for each size of the image data to becomposed do not need to be stored. Here, the size of the image data tobe composed is more than 1980×1080 pixels in the example shown in FIG.18. Even when the size of the image data to be composed is smaller than1980×1080 pixels, the scratch image data, the noise image data, and thedust image data have only to be demagnified as in the case where thesize of the image data to be composed is larger than 1980×1080 pixels.Here, the scratch image data, the noise image data, and the dust imagedata are resized (magnified) and then composed in FIG. 18. However,these three images may be resized after being composed, and composedwith the photographed image.

FIG. 19 is a flowchart showing shading processing. In FIG. 19, thespecial image processing unit 2202 causes the random seed generatingunit 2202 a and the pseudo-random number generating unit 2202 b toacquire a pseudo-random number necessary to apply shading (step S701).In the processing shown in FIG. 19, at least one pseudo-random number Ris acquired by way of example. The specific way to acquire thepseudo-random number will be described later.

After having acquired the pseudo-random number R, the special imageprocessing unit 2202 calculates a magnification factor that indicatesthe shape of shading (step S702). a is provided by, for example,(Equation 5):a=0.5+R/MAX  (Equation 5)wherein a is the magnification factor.

After having calculated the magnification factor a, the special imageprocessing unit 2202 generates a gain map in accordance with themagnification factor a (step S703). The gain map is a map having a gainsuch that the value of luminance gradually decreases with the distancefrom the pixel at the central position of a region (e.g., a region wherethe subject exists) to which the shading is applied. The maximum valueof the gain is 1. In the present embodiment, the gain map thus generatedthat is resized by the magnification factor a is used as the final gainmap.

After having generated the gain map, the special image processing unit2202 multiples the gain indicated by the gain map by the correspondingpixel (step S704). In this way, the shading effect such that theperiphery of the image data is darker is applied. Here, in the presentembodiment, the magnification factor a changes between 0.5 and 1.5depending on the value of the pseudo-random number R. In accordance withthe change of a, the shape (size) of the shading changes. FIG. 20A showsan example of a shading effect which is applied when the magnificationfactor a is 0.5. FIG. 20B shows an example of a shading effect which isapplied when the magnification factor a is 1.5.

Here, the user may feel uncomfortable if the shape of the shadingexcessively changes. Therefore, the value of the magnification factormay be controlled so that the difference between the magnificationfactor in the previous shading application processing and themagnification factor in the current shading application processing iswithin a predetermined range. For example, factor a is decided so thatthe condition in (Equation 6) is satisfied:|b−a|<0.1  (Equation 6),

wherein b is the magnification factor in the previous shadingapplication processing, and a is the magnification factor in the currentshading application processing.

In the example in the flowchart of FIG. 19, the magnification factor ais always changed. This is not a limitation. The magnification factor amay be changed only once in several frames. In this case, for the framefor which the magnification factor a has not been changed, themagnification factor a may be calculated by the linear interpolationbetween the current frame and the previous frame.

FIG. 21 is a flowchart showing processing to apply the granular noiseeffect. In FIG. 21, the special image processing unit 2202 causes therandom seed generating unit 2202 a and the pseudo-random numbergenerating unit 2202 b to acquire a pseudo-random number necessary toapply granular noise (step S801). In the processing shown in FIG. 21, atleast two pseudo-random numbers Wr and Hr are acquired. The specific wayto acquire the pseudo-random numbers will be described later. Afterhaving acquired the pseudo-random numbers Wr and Hr, the special imageprocessing unit 2202 reads the noise image data (step S802). The specialimage processing unit 2202 then calculates a cutout position of thenoise image data (step S803). The cutout position is a referentialposition to decide a cutout range of the noise image data, andcorresponds to upper left coordinates of the noise image data. In thepresent embodiment, the cutout range of the noise image data is randomlyupdated at the time of the superposition of the noise image data toreproduce the randomness of the noise resulting from the dust. Thecutout position (x, y) after update is provided by, for example,(Equation 7):x=(Wn−Wi)×Wr/MAXy=(Hn−Hi)×Hr/MAX  (Equation 7)wherein Wn is the size (the number of pixels) of the image data to becomposed in the lateral direction, Hn is the size (the number of pixels)of the image data to be composed in the longitudinal direction, Wi isthe size (the number of pixels) of the cutout range in the lateraldirection, and Hi is the size (the number of pixels) of the cutout rangein the longitudinal direction. FIG. 22 shows the relation between Wn,Hn, Wi, and Hi.

After having calculated the cutout position, the special imageprocessing unit 2202 causes the composing unit 2202 d to compose thenoise image data with the image data to be composed (step S804). Thespecial image processing unit 2202 then finishes the processing in FIG.21. The noise image data is composed as in the film noise effectapplication processing. When necessary, the size of the noise image datais adjusted to the size of the image data to be composed, and these dataare multiplied and composed.

Here, the cutout position is always changed in the example in theflowchart of FIG. 21. This is not a limitation. The cutout position mayonly be changed once in several frames or may be randomly changed. Whenthe cutout position is randomly changed, another pseudo-random number isacquired in step S801. A determination similar to, for example, that instep S606 has only to be then made.

Now, the pseudo-random number acquiring processing is described. FIG. 23is a flowchart showing the pseudo-random number acquiring processing. InFIG. 23, the special image processing unit 2202 determines whether thecurrent special image processing is the processing for a still image(step S901). Information regarding whether the current special imageprocessing is the processing for a still image is provided by, forexample, the microcomputer 234.

In the example described above, the cutout position, for example, isalways randomly updated in the special image processing for a stillimage. Therefore, when it is determined in step S901 that the currentspecial image processing is the processing for a still image, thespecial image processing unit 2202 causes the random seed generatingunit 2202 a to generate a random seed (step S902). The random seed isgenerated in accordance with various parameters associated with thedigital camera 1, such as the RAW data, camera settings, and conditionsduring photography. The specific way to generate the random seed will bedescribed later. After the generation of the random seed, the specialimage processing unit 2202 initializes the pseudo-random numbergenerator of the pseudo-random number generating unit 2202 b (stepS903). In this processing, the pseudo-random number generating unit 2202b sets the initial value to be set in the pseudo-random number generatorto the value of the random seed generated in step S902. The specialimage processing unit 2202 then causes the pseudo-random numbergenerating unit 2202 b to generate a necessary number of pseudo-randomnumber sequences (e.g., 10 random number sequences R[0] to R[9] in thecase of FIG. 10) (step S904). In this processing, the pseudo-randomnumber generating unit 2202 b uses the pseudo-random number generator togenerate the pseudo-random number sequences. The pseudo-random numbergenerating method is not particularly limited.

When it is determined in step S901 that the current special imageprocessing is not the processing for a still image, the special imageprocessing unit 2202 determines whether the current image data to becomposed is the initial frame (step S905). When it is determined in stepS905 that the current image data to be composed is the initial frame,the special image processing unit 2202 shifts the processing to stepS902. No random seed is generated yet at the point of the initial frame.Therefore, a random seed is generated in step S902. When it isdetermined in step S905 that the current image data to be composed isnot the initial frame, the special image processing unit 2202 shifts theprocessing to step S904. In this case, pseudo-random number sequencesare generated in accordance with the setting of the pseudo-random numbergenerator for the previous frame.

Now, an example of a random seed generating method is described. FIG. 24is a conceptual diagram showing the flow of a procedure for generating arandom seed by use of the RAW data to perform the special imageprocessing. In the example of FIG. 24, a random seed is generated inaccordance with the RAW data which is obtained by photography using themechanical shutter or photography using the electronic shutter. By wayof example, the value of particular coordinates of the RAW data is therandom seed. The particular coordinates are, for example, the upper leftcoordinates of the RAW data or the coordinates of the center. Otherwise,the values of the coordinates of the RAW data may be combined togenerate a random seed. For example, the result of adding up orsubtracting the values of the coordinates of the RAW data can be arandom seed, or the result of the exclusive OR of the values of thecoordinates can be a random seed. Moreover, the camera settings can becombined with the RAW data to generate a random seed. For example, anumerical value may be allocated to a camera setting such as the whitebalance mode, the setting of the luminance changing processing, thecontrast setting, and the sharpness setting, and this numerical valuemay be added to or subtracted from the random seed generated by the RAWdata to produce a final random seed. Alternatively, an image processingparameter may be combined as in FIG. 25 described later to generate arandom seed.

When the pseudo-random number generator is initialized in accordancewith the random seed generated as in FIG. 24 to generate pseudo-randomnumber sequences, a different pseudo-random number sequence can begenerated each time RAW data is obtained by photography because the RAWdata obtained by photography generally varies for every photograph.Therefore, an effect that varies for each photograph can be applied tothe image. Since the random seed also varies with the variation of thecamera settings, an effect that varies in each camera setting can beapplied to the image even in the case of the same RAW data.

FIG. 25 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of RAW data demagnified by the RAWresizing unit 210 to perform the special image processing. In theexample of FIG. 25, data regarding particular coordinates of thedemagnified RAW data is a random seed by way of example. The particularcoordinates are, for example, the upper left coordinates of the RAW dataor the coordinates of the center. Otherwise, data regarding thecoordinates of the demagnified RAW data may be combined to generate arandom seed. For example, the result of adding up or subtracting thedata regarding the coordinates of the RAW data can be a random seed, orthe result of the exclusive OR of the data regarding the coordinates canbe a random seed. Moreover, an image processing parameter may becombined with the demagnified RAW data to generate a random seed. Forexample, an image processing parameter such as the white balance gain, asaturation/color tone correction coefficient, a gamma value, an edgeenhancement degree, or noise reduction (NR) intensity may be added to orsubtracted from the random seed generated by the RAW data to produce afinal random seed. Alternatively, an image processing parameter such asthe compression rate of a recorded image or the size of a recorded imagemay be used.

When data regarding several pixels of the demagnified RAW data are usedto generate a random seed, it is possible to indirectly use informationof more than several pixels to generate a random seed in the case of theRAW data before demagnification. That is, the demagnification isperformed by the interpolation processing, so that it can be consideredthat data regarding certain coordinates in the interpolated RAW dataincludes information regarding the RAW data on the coordinates in theRAW data before demagnification. A random seed is thus generated basedon the demagnified RAW data including more information than beforedemagnification, so that even if a photography condition such as thebrightness of a scene is only slightly different, a different effect iseasily applied to the image.

FIG. 26 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of a random number generated duringphotography to perform the special image processing. The random numbergenerated during photography is generated in, for example, thepseudo-random number generating unit 2202 b. Here, the random numbergenerated during photography does not always have to be thepseudo-random number. The random number generated during photography maybe generated in a pseudo-random number generating unit different fromthe pseudo-random number generating unit 2202 b. Here, in the example ofFIG. 26 as well, the RAW data and the demagnified RAW data may becombined, or the camera setting or the image processing parameter may becombined to generate a final pseudo-random number.

As shown in FIG. 26, it is possible to apply an effect that varies foreach photograph to the image by generating a random number for eachphotograph and generating a random seed in accordance with the generatedrandom number.

FIG. 27 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use of the YC data (hereinafter referred toas intermediate YC data) obtained by subjecting the RAW data to thebasic image processing to perform the special image processing. By wayof example, the value of particular coordinates of the intermediate YCdata is the random seed. The particular coordinates are, for example,the upper left coordinates of the intermediate YC data or thecoordinates of the center. Otherwise, the values of the coordinates ofthe intermediate YC data may be combined to generate a random seed.

When the intermediate YC data is used to generate a random seed as shownin FIG. 27, it is possible to obtain advantageous effects similar to theadvantageous effects obtained when a random seed is generated inconsideration of the camera setting or the image processing parameter.

FIG. 28 is a conceptual diagram showing the flow of a procedure forgenerating a random seed by use various conditions during photography toperform the special image processing. In the example of FIG. 25, anexposure condition, a subject condition, and a camera state, forexample, are used as the conditions for photography. The exposurecondition is, for example, at least one of the shutter speed, theaperture value, and the ISO. The subject condition is, for example, thepresence of a subject such as a face or a pet, the size, the position,and the number of, if any, subjects. The camera state is, for example,the temperature of the image pickup device 204, the internal temperatureof the camera body 200, the remaining capacity of a battery, the kind,capacity, and remaining capacity of the recording medium 232, a focusposition, and a zoom position, during photography. The exposurecondition, the subject condition, and the camera state are converted tonumerical values (when these are originally numerical values, thesevalues can be used) if necessary, and the converted conditions are addedor subtracted to generate a random seed. Here, in the example of FIG. 28as well, image data such as the RAW data, the demagnified RAW data, andthe intermediate YC data may be combined, or the camera setting or theimage processing parameter may be combined to generate a finalpseudo-random number.

As shown in FIG. 28, it is also possible to apply an effect that variesfor each photograph to the image by generating a random seed usingvarious conditions during photography. Here, as described above, thepseudo-random number sequence is characterized in that the same sequenceis generated from the same random seed. Therefore, if information togenerate a random seed is recorded, it is possible to perform thespecial image processing to later apply the noise effect to the imagedata for which the special image processing to apply the noise effecthas not been performed at the time of the image processing duringphotography. FIG. 29A and FIG. 29B are diagrams showing file structuresof image files when information for generating a random seed is recordedin an image file. FIG. 29A shows an example of a still image filerecorded in the JPEG format (referred to as a JPEG file), and FIG. 29Bshows an example of a RAW file. Moving image files are not shown. Amoving image file has special structures as regards its image data andheader information. There is no difference in the manner of recordinginformation to generate a random seed between the moving image file andthe still image file.

As shown in FIG. 29A, the JPEG file has a header recording portion, athumbnail JPEG data recording portion, a main image JPEG data recordingportion, and a screen-nail JPEG data recording portion.

The header recording portion is a recording portion for recording, asmetadata, various kinds of information such as the exposure condition,the subject condition, and the camera state. FIG. 29A shows an exampleof how to record the exposure condition, the subject condition, and thecamera state. In addition to the above information, information such asthe camera setting and the image processing parameter may be recorded.Moreover, when a pseudo-random number is generated, the generatedpseudo-random number may be directly recorded.

The thumbnail JPEG data recording portion is a recording portion forrecording thumbnail display image data for still image reproductionafter compression in the JPEG format. The main image JPEG data recordingportion is a recording portion for recording still image data obtainedby photography using the mechanical shutter 202 after compression in theJPEG format. The screen-nail JPEG data recording portion is a recordingportion for recording screen-nail display image data after compressionin the JPEG format.

As shown in FIG. 29B, the RAW file has a header recording portion, athumbnail JPEG data recording portion, a RAW data recording portion, ademagnified RAW data recording portion, and a screen-nail JPEG datarecording portion. The header recording portion is a recording portionfor recording, as metadata, various kinds of information such as theexposure condition, the subject condition, and the camera state. FIG.29B shows an example of how to record the exposure condition, thesubject condition, and the camera state. There is little differencebetween the header recording portion of the JPEG file and that of theRAW file.

The thumbnail JPEG data recording portion is a recording portion forrecording thumbnail display image data for RAW reproduction aftercompression in the JPEG format. The RAW data recording portion is arecording portion for recording RAW data obtained by photography usingthe mechanical shutter 202 or photography using the electronic shutter.The demagnified RAW data recording portion is a recording portion forrecording demagnified RAW data necessary to generate a random seed byusing the demagnified RAW data shown in FIG. 25. The screen-nail JPEGdata recording portion is a recording portion for recording screen-naildisplay image data after compression in the JPEG format.

Now, the reproduction processing is described. FIG. 30 is a flowchartshowing the reproduction processing. In FIG. 30, the microcomputer 234displays a list of image files recorded in the recording medium 232(step S1101). In this processing, the microcomputer 234 reads thethumbnail JPEG data for the image files into the SDRAM 214. Themicrocomputer 234 then inputs the read thumbnail JPEG data to the imagecompressing/decompressing unit 228. The image compressing/decompressingunit 228 decompresses the input thumbnail JPEG data, and inputs thedecompressed thumbnail JPEG data to the display driver 224. The displaydriver 224 displays the list of the image files on the display unit 226based on the input thumbnail JPEG data.

After the display of the list, the microcomputer 234 determines whetherto finish the reproduction processing (step S1102). For example, whenthe reproduction button is pressed again, the microcomputer 234determines to finish the reproduction. When it is determined in stepS1102 to finish the reproduction processing, the microcomputer 234finishes the processing in FIG. 30.

When it is determined in step S1102 not to finish the reproductionprocessing, the microcomputer 234 waits for the user to select an imagefile (step S1103). When the user operates the operating unit 236 toselect an image file on the displayed list, the microcomputer 234determines whether the selected image file is a moving image file (stepS1104).

When it is determined in step S1104 that the selected image file is amoving image file, the microcomputer 234 reads the number of frames inmoving image data recorded in the selected moving image file (stepS1105). The microcomputer 234 then initializes the count value i of thenumber of frames in the moving image data to be reproduced (step S1106).The initial value of the count value is, for example, 1.

The microcomputer 234 then reproduces the i-th frame in the moving imagedata recorded in the selected moving image file, and displays the frameon the display unit 226 (step S1107). In this processing, themicrocomputer 234 reads the i-th frame in the moving image data recordedin the moving image file selected by the user into the SDRAM 214. Themicrocomputer 234 then inputs the read moving image data of the i-thframe to the image compressing/decompressing unit 228. The imagecompressing/decompressing unit 228 decompresses the input moving imagedata, and inputs the decompressed moving image data to the displaydriver 224. The display driver 224 displays an image corresponding tothe input moving image data of the i-th frame on the display unit 226.

After the reproduction and display of the i-th frame of the moving imagedata, the microcomputer 234 adds 1 to the count value (step S1108). Themicrocomputer 234 then determines whether the count value i is equal toor lower than the number of frames, that is, whether the reproduction ofall the frames has finished (step S1109). When the count value i isequal to or lower than the number of frames in step S1109, frames to bereproduced still remain. In this case, the microcomputer 234 returns theprocessing to step S1107 and then reproduces and displays the nextframe. When the count value i is higher than the number of frames instep S1109, this means that the reproduction of moving image file hasfinished. In this case, the microcomputer 234 returns the processing tostep S1101.

When it is determined in step S1104 that the selected image file is nota moving image file, that is, a still image file, the microcomputer 234reads still image data recorded in the selected still image file (stepS1110). The microcomputer 234 then reproduces the read still image data(when the file structure is the file structure shown in FIG. 29A or FIG.29B, the still image data is the main image JPEG in the case of JPEG orthe screen-nail JPEG in the case of RAW), and displays the still imagedata on the display unit 226 (step S1111). In this processing, themicrocomputer 234 inputs the read still image data to the imagecompressing/decompressing unit 228. The image compressing/decompressingunit 228 decompresses the input still image data, and inputs thedecompressed still image data to the display driver 224. The displaydriver 224 displays an image corresponding to the input still image dataon the display unit 226.

The microcomputer 234 then determines whether the user has performed anediting operation (step S1112). The editing operation is an operationfor the user to select an item to apply the noise effect from the menu.In this case, the operating unit 236 necessary for the selectionoperation functions as an example of an instruction unit. Editingoperations to apply effects other than the noise effect may be similarto conventional editing operations, and are therefore not described.

When it is determined in step S1112 that the editing operation has notbeen performed, the microcomputer 234 determines whether to finish thedisplay of the still image (step S1113). For example, when the menubutton is pressed by the user, the microcomputer 234 determines tofinish the display. When it is determined in step S1113 not to finishthe display, the microcomputer 234 returns the processing to step S1112.In this case, the display of the still image is continued. When it isdetermined in step S1113 to finish the display, the microcomputer 234finishes the processing in FIG. 30.

When it is determined in step S1112 that the editing operation has beenperformed, the microcomputer 234 performs editing processing (stepS1114). The editing processing is further described below with referenceto FIG. 31. Here, the processing in step S1114 is the processing for astill image file. However, the processing described below is alsoapplicable to a moving image file.

FIG. 31 is a flowchart showing the editing processing. In FIG. 31, themicrocomputer 234 reads a still image file to be currently edited (stepS1201). The microcomputer 234 then determines whether the read stillimage file is a RAW file (step S1202). When it is determined in stepS1202 that the still image file is a RAW file, the microcomputer 234performs image processing for this RAW file (step S1203). The imageprocessing in step S1203 is the same as the image processing shown inFIG. 4. Since the still image file is a RAW file, both the basic imageprocessing and the special image processing are performed. Here, apseudo-random number necessary for the special image processing can begenerated from a random seed which is generated by the combination ofthe RAW data (or the demagnified RAW data), the camera setting, and theimage processing parameter, as described above. In this case, the camerasetting recorded in the header recording portion of the RAW file may beused, or the camera setting newly set by the user at the time of theediting processing may be used. When the camera setting recorded in theheader recording portion is used, the same noise effect as that duringphotography can be applied. On the other hand, when the camera settingnewly set by the user is used, the noise effect corresponding to thechange can be applied. When it is determined in step S1202 that thestill image file is not a RAW file, that is, the still image file is aJPEG file, the microcomputer 234 performs special image processing forthis JPEG file (step S1204). The special image processing in step S1204is the same as the special image processing shown in FIG. 6. Here, apseudo-random number necessary for the special image processing can begenerated from a random seed which is generated by the combination ofvarious conditions during photography, the camera setting, and the imageprocessing parameter, as described above. In this case, the camerasetting recorded in the header recording portion of the JPEG file may beused, or the camera setting newly set by the user at the time of theediting processing may be used. When the camera setting recorded in theheader recording portion is used, the same noise effect as that duringphotography can be applied. On the other hand, when the camera settingnewly set by the user is used, the noise effect corresponding to thechange can be applied.

As described above, in the present embodiment, parts of the scratchimage data and the noise image data are randomly cut out, and thescratch image data, the noise image data, and the dust image data arethen composed to generate composite noise image data. Further, thecomposite noise image data is composed with the image data to becomposed. Thus, a specific noise shown during film photography isdecomposed and then composed in the present embodiment, so that anatural sense of noise can be provided to the user without the recordingof long-time moving image data.

The scratch image data is demagnified and then recorded by takingadvantage of the fact that the scratch image data is characterized bybeing highly correlated in one direction. As a result, it is possible tosave the capacity to record the scratch image data.

When the sizes of the scratch image data and the noise image data do notcorrespond to the size of the image data to be composed, processing isperformed so that the sizes of the scratch image data and the noiseimage data correspond to the size of the image data to be composed. As aresult, the scratch image data and the noise image data do not need tobe recorded any longer for each size of the image data to be composed.

In the present embodiment, a random seed is generated from the RAW dataobtained by photography, the camera setting during photography, and theimage processing parameter, and pseudo-random numbers to apply the noiseeffect, the shading effect, and the granular noise effect are generatedin accordance with the random seed. As a result, the effect suited tothe situation during photography can be applied.

Here, in the present embodiment, three kinds of data: the scratch imagedata, the noise image data, and the dust image data are composed withthe image data to be composed. Actually, not all the image data need tobe composed. For example, the dust image data may not be composed.

The method of each process performed by the imaging apparatus in theembodiment described above, that is, the processing shown in eachflowchart can be stored as a program executable by the microcomputer234. Otherwise, the program can be stored and distributed in a storagemedium of an external storage device such as a memory card (e.g., a ROMcard, a RAM card), a magnetic disk (e.g., a floppy disk, a hard disk),an optical disk (e.g., a CD-ROM, a DVD), or a semiconductor memory. Themicrocomputer 234 then reads the program stored in the storage medium ofthe external storage device, and the operation of the microcomputer 234is controlled by the read program, so that the microcomputer 234 canperform the processing described above.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An imaging apparatus comprising: an imaging unitwhich photographs a subject to obtain image data; a storage unit whichstores film noise effect generating image data, wherein the film noiseeffect generating image data includes scratch image data, noise imagedata and dust image data, the scratch image data and the noise imagedata are larger than the image data, and the film noise effect imagedata is obtained by combining the scratch image data, noise image dataand dust image data multiplied by a gain in the range of 0 to 1; arandom seed generating unit which generates a random seed to decide apseudo-random number sequence; a pseudo-random number generating unitwhich generates a pseudo-random number in accordance with the generatedrandom seed; a cut out range deciding unit which decides a cut out rangeof the film noise effect generating image data based on the generatedpseudo-random number, wherein a lateral size of the cut out range is thesame as that of the image data; and a special image processing unitwhich cuts out a part of the film noise effect generating image databased on the cut out range, resizes a longitudinal size of the filmnoise effect image data in the cut out range in accordance with alongitudinal size of the image data, and combines the resized film noiseeffect image data with the image data, wherein the random seed isdecided during photography.
 2. The imaging apparatus according to claim1, further comprising an instruction unit which instructs to performspecial image processing for the image data, wherein the special imageprocessing unit subjects the image data to the special image processingin accordance with the instruction from the instruction unit.
 3. Theimaging apparatus according to claim 1, wherein the random seedgenerating unit generates the random seed based on at least one of theimage data, an exposure condition during photography, a subjectcondition during photography, a camera state during photography, and arandom number calculated during photography.
 4. The imaging apparatusaccording to claim 3, wherein the random seed generating unit generatesthe random seed further based on an image processing parameter used inthe image processing provided to the image data.
 5. The imagingapparatus according to claim 4, wherein the image processing parameterincludes at least one of a white balance setting in the image data, acontrast setting in the image data, a saturation setting in the imagedata, an edge enhancement degree setting in the image data, a noisereduction intensity setting in the image data, a compression rate of theimage data, and the size of the image data.
 6. The imaging apparatusaccording to claim 4, wherein the random seed generating unit generatesthe random seed by use of intermediate image data which is obtained byimage processing of the image data based on the image processingparameter.
 7. The imaging apparatus according to claim 3, wherein therandom seed generating unit generates the random seed by use ofdemagnified image data which is obtained by demagnifying the image data.8. An image processing apparatus comprising: a storage unit which storesfilm noise effect generating image data, wherein the film noise effectgenerating image data includes scratch image data, noise image data anddust image data, the scratch image data and the noise image data arelarger than subject image data obtained from an imaging unit, and thefilm noise effect image data is obtained by combining the scratch imagedata, noise image data and dust image data multiplied by a gain in therange of 0 to 1; a random seed generating unit which generates a randomseed to decide a pseudo-random number sequence; a pseudo-random numbergenerating unit which generates a pseudo-random number in accordancewith the generated random seed; a cut out range deciding unit whichdecides a cut out range of the film noise effect generating image databased on the generated pseudo-random number, wherein a lateral size ofthe cut out range is the same as that of the subject image data; and aspecial image processing unit which cuts out a part of the film noiseeffect generating image data based on the cut out range, resizes alongitudinal size of the film noise effect image data in the cut outrange in accordance with a longitudinal size of the subject image data,and combines the resized film noise effect image data with the subjectimage data, wherein the random seed is decided during the acquisition ofthe subject image data.
 9. The image processing apparatus according toclaim 8, further comprising an instruction unit which instructs toperform special image processing for the subject image data, wherein thespecial image processing unit subjects the subject image data to thespecial image processing in accordance with the instruction from theinstruction unit.
 10. The image processing apparatus according to claim8, wherein the random seed generating unit generates the random seedbased on at least one of the subject image data, an exposure conditionduring photography, a subject condition during photography, a camerastate during photography, and a random number calculated duringphotography.
 11. The image processing apparatus according to claim 10,wherein the random seed generating unit generates the random seedfurther based on an image processing parameter used in the imageprocessing provided to the subject image data.
 12. The image processingapparatus according to claim 11, wherein the random seed generating unitgenerates the random seed by use of intermediate image data which isobtained by image processing of the subject image data based on theimage processing parameter.
 13. The image processing apparatus accordingto claim 10, wherein the random seed generating unit generates therandom seed by use of demagnified image data which is obtained bydemagnifying the subject image data.
 14. An image processing methodcomprising: storing film noise effect generating image data, wherein thefilm noise effect generating image data includes scratch image data,noise image data and dust image data, the scratch image data and thenoise image data are larger than subject image data obtained from animaging unit, and the film noise effect image data is obtained bycombining the scratch image data, noise image data and dust image datamultiplied by a gain in the range of 0 to 1; generating a random seed byuse of information obtained during an acquisition of image data inresponse to an instruction to perform special image processing for thesubject image data; generating a pseudo-random number in accordance withthe generated random seed; determining a cut out range of the film noiseeffect generating image data based on the generated pseudo-randomnumber, wherein a lateral size of the cut out range is the same as thatof the subject image data; cutting out a part of the film noise effectgenerating image data based on the cut out range; resizing alongitudinal size of the film noise effect image data in the cut outrange in accordance with a longitudinal size of the subject image datato generate resized film noise effect image data; and combining theresized film noise effect image data with the subject image data. 15.The image processing method according to claim 14, further comprisinginstructing to perform special image processing for the subject imagedata, wherein the special image processing special image processing isperformed in accordance with the instruction.
 16. The image processingmethod according to claim 14, wherein the random seed is generated basedon at least one of the subject image data, an exposure condition duringphotography, a subject condition during photography, a camera stateduring photography, and a random number calculated during photography.17. The image processing method according to claim 16, wherein therandom seed is generated further based on an image processing parameter.18. The imaging processing method according to claim 17, wherein therandom seed is generated by use of intermediate image data which isobtained by image processing of the subject image data based on theimage processing parameter.
 19. The imaging processing method accordingto claim 16, wherein the random seed is generated by use of demagnifiedimage data which is obtained by demagnifying the subject image data. 20.A non-transitory recording medium on which an image processing programcausing a computer to execute: storing film noise effect generatingimage data, wherein the film noise effect generating image data includesscratch image data, noise image data and dust image data, the scratchimage data and the noise image data are larger than subject image dataobtained from an imaging unit, and the film noise effect image data isobtained by combining the scratch image data, noise image data and dustimage data multiplied by a gain in the range of 0 to 1; generating arandom seed by use of information obtained during an acquisition ofimage data in response to an instruction to perform special imageprocessing for the subject image data; generating a pseudo-random numberin accordance with the generated random seed; determining a cut outrange of the film noise effect generating image data based on thegenerated pseudo-random number, wherein a lateral size of the cut outrange is the same as that of the subject image data; cutting out a partof the film noise effect generating image data based on the cut outrange; resizing a longitudinal size of the film noise effect image datain the cut out range in accordance with a longitudinal size of thesubject image data to generate resized film noise effect image data; andcombining the resized film noise effect image data with the subjectimage data.